[TRTLLM-13948][test] Add DeepSeek R1/V3.2/V3-Lite disaggregated accur… by asfiyab-nvidia · Pull Request #16344 · NVIDIA/TensorRT-LLM

asfiyab-nvidia · 2026-07-13T23:16:04Z

This pull request adds new integration tests for the DeepSeekR1 and DeepSeekV3.2 models to validate the behavior of KV cache manager v1 using the NIXL Python transceiver in a disaggregated serving setup.

New test coverage:

Added test_kv_cache_v2_nixl_python in TestDeepSeekR1 and TestDeepSeekV32Exp to test DeepSeekR1 and DeepSeekV3.2 with NIXL Python transceiver and KV cache manager v1, including configuration for context and generation servers with tensor parallelism and cache settings.
Test suite and test database updates:

Registered the new parameterized tests for cache manager v1 in the QA test list (test_lists/qa/llm_function_core.txt).
Added the new tests to the l0_dgx_b300 test database for automated runs on the appropriate hardware.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

Tests

Added new integration test coverage for KV cache v2 functionality with disaggregated serving setup
Extended test scenarios for NIXL transceiver in Python mode
Updated test configurations to include cache manager v1 parameterization variants for enhanced compatibility testing

asfiyab-nvidia · 2026-07-13T23:16:15Z

/bot run --add-multi-gpu-test --disable-fail-fast

tensorrt-cicd · 2026-07-13T23:32:47Z

PR_Github #59041 [ run ] triggered by Bot. Commit: 648b5a5 Link to invocation

tensorrt-cicd · 2026-07-14T03:09:29Z

PR_Github #59041 [ run ] completed with state SUCCESS. Commit: 648b5a5
/LLM/main/L0_MergeRequest_PR pipeline #47570 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

asfiyab-nvidia · 2026-07-14T18:01:02Z

/bot run --add-multi-gpu-test --disable-fail-fast

tensorrt-cicd · 2026-07-14T18:09:34Z

PR_Github #59251 [ run ] triggered by Bot. Commit: 6e02fbd Link to invocation

tensorrt-cicd · 2026-07-14T22:01:13Z

PR_Github #59251 [ run ] completed with state SUCCESS. Commit: 6e02fbd
/LLM/main/L0_MergeRequest_PR pipeline #47739 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

asfiyab-nvidia · 2026-07-15T01:11:13Z

The failing tests listed below are unrelated to the transceiver changes made in this PR

unittest/auto_deploy/singlegpu/models
unittest/auto_deploy/singlegpu/models/test_glm4_moe_modeling.py::test_glm4_moe_moe_equivalence[dtype0-2-6]

coderabbitai · 2026-07-15T01:18:31Z

📝 Walkthrough

Walkthrough

Updated disaggregated serving integration tests for NIXL Python KV-cache manager v1, added DeepSeekR1 coverage, adjusted DeepSeekV32Exp configuration, and registered both tests in QA and DGX B300 test lists.

Changes

Disaggregated KV-cache coverage

Layer / File(s)	Summary
Update disaggregated server configuration `tests/integration/defs/accuracy/test_disaggregated_serving.py`	Worker discovery now uses context and generation server instance counts without explicit ports or URLs.
Configure DeepSeekV32Exp cache-manager v1 test `tests/integration/defs/accuracy/test_disaggregated_serving.py`	The test adds fixed token, MoE, tensor-parallel, expert-parallel, autotuner, and overlap-scheduling settings for the cache-manager v1 path.
Add DeepSeekR1 coverage and scheduling `tests/integration/defs/accuracy/test_disaggregated_serving.py`, `tests/integration/test_lists/qa/llm_function_core.txt`, `tests/integration/test_lists/test-db/l0_dgx_b300.yml`	Adds the DeepSeekR1 NIXL Python cache-manager v1 test and registers DeepSeekR1 and DeepSeekV32Exp cases in test lists.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant AccuracyTest
  participant launch_disaggregated_llm
  participant ContextServer
  participant GenerationServer
  participant NIXLPython
  AccuracyTest->>launch_disaggregated_llm: launch cache-manager v1 configuration
  launch_disaggregated_llm->>ContextServer: start context server
  launch_disaggregated_llm->>GenerationServer: start generation server
  ContextServer->>NIXLPython: write KV cache
  NIXLPython->>GenerationServer: provide KV cache
  GenerationServer-->>AccuracyTest: return generation result

Suggested reviewers: nv-xtf, bo-nv, lfr-0531

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title is concise and matches the core change: new disaggregated accuracy tests for DeepSeek models.
Description check	✅ Passed	It includes a clear summary, checklist, and test notes, though the Description and Test Coverage sections are mostly placeholder text.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

coderabbitai

Actionable comments posted: 1

🧹 Nitpick comments (1)

tests/integration/test_lists/qa/llm_function_core.txt (1)
1-1: 📐 Maintainability & Code Quality | 🔵 Trivial

Coverage currently limited to cache_mgr_v1 only for both new entries.

Both new QA entries (TestDeepSeekR1 and TestDeepSeekV32Exp test_kv_cache_v2_nixl_python[cache_mgr_v1]) only exercise the KV-cache-manager-v1 path, per the parametrize decorators added in test_disaggregated_serving.py. This matches the PR's intentional narrowing (per commit history) after CI issues with the v2 path, but flagging for follow-up: once the cache_mgr_v2 path stabilizes, this list (and l0_dgx_b300.yml) should be updated to add the [cache_mgr_v2] variant, consistent with how TestGemma3_1BInstruct/TestGPTOSS register both cache_mgr_v1 and cache_mgr_v2 ids here.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/integration/test_lists/qa/llm_function_core.txt` at line 1, Update the
QA registrations for TestDeepSeekR1 and TestDeepSeekV32Exp to include the
test_kv_cache_v2_nixl_python[cache_mgr_v2] variant once the cache_mgr_v2 path is
stable, matching the existing cache_mgr_v1 entries and the dual registrations
used by TestGemma3_1BInstruct and TestGPTOSS; also add the corresponding variant
to l0_dgx_b300.yml.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/integration/defs/accuracy/test_disaggregated_serving.py`:
- Around line 2809-2869: Set enable_autotuner to False in both ctx_server_config
and gen_server_config within TestDeepSeekR1.test_kv_cache_v2_nixl_python.
Preserve all other server settings and ensure both context and generation
servers skip autotuner warmup.

---

Nitpick comments:
In `@tests/integration/test_lists/qa/llm_function_core.txt`:
- Line 1: Update the QA registrations for TestDeepSeekR1 and TestDeepSeekV32Exp
to include the test_kv_cache_v2_nixl_python[cache_mgr_v2] variant once the
cache_mgr_v2 path is stable, matching the existing cache_mgr_v1 entries and the
dual registrations used by TestGemma3_1BInstruct and TestGPTOSS; also add the
corresponding variant to l0_dgx_b300.yml.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 47c62172-cd90-48d0-8774-b8a2019896ad

📥 Commits

Reviewing files that changed from the base of the PR and between 54d484f and 6e02fbd.

📒 Files selected for processing (3)

tests/integration/defs/accuracy/test_disaggregated_serving.py
tests/integration/test_lists/qa/llm_function_core.txt
tests/integration/test_lists/test-db/l0_dgx_b300.yml

asfiyab-nvidia · 2026-07-15T01:33:20Z

/bot run --add-multi-gpu-test --disable-fail-fast

tensorrt-cicd · 2026-07-15T01:43:46Z

PR_Github #59320 [ run ] triggered by Bot. Commit: d555a53 Link to invocation

tensorrt-cicd · 2026-07-15T05:57:25Z

PR_Github #59320 [ run ] completed with state SUCCESS. Commit: d555a53
/LLM/main/L0_MergeRequest_PR pipeline #47805 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

asfiyab-nvidia · 2026-07-15T17:33:22Z

/bot run

tensorrt-cicd · 2026-07-15T17:39:13Z

PR_Github #59491 [ run ] triggered by Bot. Commit: d555a53 Link to invocation

tensorrt-cicd · 2026-07-15T19:40:18Z

PR_Github #59491 [ run ] completed with state SUCCESS. Commit: d555a53
/LLM/main/L0_MergeRequest_PR pipeline #47950 completed with status: 'SUCCESS'

CI Report

Link to invocation

tensorrt-cicd · 2026-07-20T19:05:10Z

PR_Github #60436 [ run ] completed with state FAILURE. Commit: a56f77b
/LLM/main/L0_MergeRequest_PR pipeline #48768 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

asfiyab-nvidia · 2026-07-20T19:14:01Z

/bot run --extra-stage "DGX_B300-4_GPUs-PyTorch-Post-Merge-1, DGX_B300-4_GPUs-PyTorch-Post-Merge-2"

tensorrt-cicd · 2026-07-20T19:22:02Z

PR_Github #60441 [ run ] triggered by Bot. Commit: a56f77b Link to invocation

tensorrt-cicd · 2026-07-20T19:57:53Z

PR_Github #60441 [ run ] completed with state FAILURE. Commit: a56f77b
/LLM/main/L0_MergeRequest_PR pipeline #48776 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

asfiyab-nvidia · 2026-07-20T20:18:02Z

/bot run --extra-stage "DGX_B300-4_GPUs-PyTorch-Post-Merge-1, DGX_B300-4_GPUs-PyTorch-Post-Merge-2"

tensorrt-cicd · 2026-07-20T20:23:27Z

PR_Github #60449 [ run ] triggered by Bot. Commit: c34b46c Link to invocation

tensorrt-cicd · 2026-07-20T22:54:48Z

PR_Github #60449 [ run ] completed with state FAILURE. Commit: c34b46c
/LLM/main/L0_MergeRequest_PR pipeline #48784 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

asfiyab-nvidia · 2026-07-20T23:17:02Z

/bot run --disable-fail-fast --extra-stage "DGX_B300-4_GPUs-PyTorch-Post-Merge-1, DGX_B300-4_GPUs-PyTorch-Post-Merge-2"

tensorrt-cicd · 2026-07-20T23:23:41Z

PR_Github #60490 [ run ] triggered by Bot. Commit: c34b46c Link to invocation

tensorrt-cicd · 2026-07-21T02:23:51Z

PR_Github #60490 [ run ] completed with state FAILURE. Commit: c34b46c
/LLM/main/L0_MergeRequest_PR pipeline #48816 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

…acy tests Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>

1. Revert changes to DeepSeekV3-Lite test 2. Update DeepSeekR1 and DeepSeekV3.2 tests to test only cache_mgr_v1 Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>

Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>

nv-xtf · 2026-07-21T06:52:58Z

/bot run --disable-fail-fast --extra-stage "DGX_B300-4_GPUs-PyTorch-Post-Merge-1, DGX_B300-4_GPUs-PyTorch-Post-Merge-2"

tensorrt-cicd · 2026-07-21T06:58:10Z

PR_Github #60617 [ run ] triggered by Bot. Commit: 16ae92b Link to invocation

tensorrt-cicd · 2026-07-21T09:57:03Z

PR_Github #60617 [ run ] completed with state FAILURE. Commit: 16ae92b
/LLM/main/L0_MergeRequest_PR pipeline #48923 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

nv-xtf · 2026-07-21T10:18:40Z

/bot run --disable-fail-fast --extra-stage "DGX_B300-4_GPUs-PyTorch-Post-Merge-1, DGX_B300-4_GPUs-PyTorch-Post-Merge-2"

tensorrt-cicd · 2026-07-21T10:24:51Z

PR_Github #60673 [ run ] triggered by Bot. Commit: 16ae92b Link to invocation

tensorrt-cicd · 2026-07-21T12:07:56Z

PR_Github #60673 [ run ] completed with state FAILURE. Commit: 16ae92b
/LLM/main/L0_MergeRequest_PR pipeline #48967 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

nv-xtf · 2026-07-21T14:12:16Z

/bot run --disable-fail-fast --extra-stage "DGX_B300-4_GPUs-PyTorch-Post-Merge-1, DGX_B300-4_GPUs-PyTorch-Post-Merge-2"

tensorrt-cicd · 2026-07-21T14:18:32Z

PR_Github #60706 [ run ] triggered by Bot. Commit: 16ae92b Link to invocation

nv-xtf · 2026-07-21T14:38:47Z

/bot run --stage-list "DGX_B300-4_GPUs-PyTorch-Post-Merge-1, DGX_B300-4_GPUs-PyTorch-Post-Merge-2"

tensorrt-cicd · 2026-07-21T14:44:29Z

PR_Github #60711 [ run ] triggered by Bot. Commit: 16ae92b Link to invocation

tensorrt-cicd · 2026-07-21T14:44:33Z

PR_Github #60706 [ run ] completed with state ABORTED. Commit: 16ae92b

Link to invocation

tensorrt-cicd · 2026-07-21T17:12:26Z

PR_Github #60711 [ run ] completed with state FAILURE. Commit: 16ae92b
/LLM/main/L0_MergeRequest_PR pipeline #49000 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

asfiyab-nvidia requested review from a team as code owners July 13, 2026 23:16

asfiyab-nvidia marked this pull request as draft July 13, 2026 23:16

github-actions Bot assigned asfiyab-nvidia Jul 13, 2026

Shixiaowei02 requested a review from nv-xtf July 14, 2026 05:32

asfiyab-nvidia marked this pull request as ready for review July 15, 2026 01:08

asfiyab-nvidia requested review from a team as code owners July 15, 2026 01:08

asfiyab-nvidia requested review from BowenFu, brnguyen2, mlefeb01 and mzweilz July 15, 2026 01:08

coderabbitai Bot reviewed Jul 15, 2026

View reviewed changes

Comment thread tests/integration/defs/accuracy/test_disaggregated_serving.py

mzweilz approved these changes Jul 15, 2026

View reviewed changes

asfiyab-nvidia requested a review from chienchunhung July 15, 2026 17:51

asfiyab-nvidia force-pushed the dev-asfiyab-nvidia/deepseek-r1-accuracy branch from a56f77b to c34b46c Compare July 20, 2026 20:17

asfiyab-nvidia added 5 commits July 21, 2026 14:44

[TRTLLM-13948][test] Add DeepSeek R1/V3.2/V3-Lite disaggregated accur…

2caebc7

…acy tests Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>

changes:

bf87cb9

1. Revert changes to DeepSeekV3-Lite test 2. Update DeepSeekR1 and DeepSeekV3.2 tests to test only cache_mgr_v1 Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>

disable autotuner for DeepSeekR1 test

02257d0

Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>

address comments

ab15a80

Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>

pre-commit run formatting

16ae92b

Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>

nv-xtf force-pushed the dev-asfiyab-nvidia/deepseek-r1-accuracy branch from c34b46c to 16ae92b Compare July 21, 2026 06:44

nv-xtf mentioned this pull request Jul 21, 2026

[TRTLLM-13948][test] Migrate DeepSeek R1/V3.2 disagg perf cases to transceiver V2 #16669

Draft

1 task

Uh oh!

Conversation

asfiyab-nvidia commented Jul 13, 2026 • edited by nv-xtf Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Test Coverage

PR Checklist

GitHub Bot Help

Tests

Uh oh!

asfiyab-nvidia commented Jul 13, 2026

Uh oh!

tensorrt-cicd commented Jul 13, 2026

Uh oh!

tensorrt-cicd commented Jul 14, 2026

Uh oh!

asfiyab-nvidia commented Jul 14, 2026

Uh oh!

tensorrt-cicd commented Jul 14, 2026

Uh oh!

tensorrt-cicd commented Jul 14, 2026

Uh oh!

asfiyab-nvidia commented Jul 15, 2026

Uh oh!

coderabbitai Bot commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

asfiyab-nvidia commented Jul 15, 2026

Uh oh!

tensorrt-cicd commented Jul 15, 2026

Uh oh!

tensorrt-cicd commented Jul 15, 2026

Uh oh!

asfiyab-nvidia commented Jul 15, 2026

Uh oh!

tensorrt-cicd commented Jul 15, 2026

Uh oh!

tensorrt-cicd commented Jul 15, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

asfiyab-nvidia commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

asfiyab-nvidia commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

asfiyab-nvidia commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 20, 2026

Uh oh!

tensorrt-cicd commented Jul 21, 2026

Uh oh!

nv-xtf commented Jul 21, 2026

Uh oh!

tensorrt-cicd commented Jul 21, 2026

Uh oh!

tensorrt-cicd commented Jul 21, 2026

Uh oh!

nv-xtf commented Jul 21, 2026

Uh oh!

tensorrt-cicd commented Jul 21, 2026

Uh oh!

tensorrt-cicd commented Jul 21, 2026

Uh oh!

nv-xtf commented Jul 21, 2026

Uh oh!

asfiyab-nvidia commented Jul 13, 2026 •

edited by nv-xtf

Loading

coderabbitai Bot commented Jul 15, 2026 •

edited

Loading